Randomized Controlled Trials (I)

Adam La Caze

Semester 1, 2023

Objectives

  • Be able to critically appraise a randomized controlled trial

  • Be able to identify important sources of bias in randomized trials

  • Be able to describe the methods used to reduce sources of bias in randomized trials

Key concepts

  • It is useful to take a structured approach to critical appraisal—but also take the time to think about the study and consider additional sources of bias

  • Intention-to-treat analyses compare participants according to how they were randomized (even if they didn’t take the treatment). It seeks to reduce bias that can occur due to differential drop-out.

  • Masking (or “blinding”) can occur at the level of participant, investigator, data collector, and clinician. Masking seeks to reduce biases that occur if participants/investigators know which treatment a participant is receiving

Critical Appraisal

Are the results of the study valid?

Internal Validity

the degree to which the results of the study are accurate for the participants in the trial

External Validity

the degree to which the results of the study are accurate for patients receiving the treatment in routine care

Good advice for critical appraisal:

Read the study and think about it.

First: Determine the PICO for the study.

PICO for critical appraisal
Critical Appraisal
Participants Who was recruited to the study? What were the inclusion/exclusion criteria? Who participated?
Intervention What intervention did the treatment group receive?
Comparator What did the control or placebo group receive?
Outcome What was the primary outcome of the trial?

Second: Methods

  1. Was the assignment of patients to treatments randomized?

  2. Were the groups similar at the start of the trial?

  3. Aside from allocated treatment, were groups treated equally?

  1. Were all patients who entered the trial accounted for?—and were they analysed in the groups to which they were randomized?

  2. Were measures objective or were the patients and clinicians kept “blind” to which treatment was being received?

Third: Results

  1. How large was the treatment effect? (How was the primary endpoint and important second endpoints measured?)

  2. How precise was the estimate of the treatment effect?

Fourth: External validity

  1. Will the results help me in caring for my patient?

Critical Appraisal: Worked Example

CURE study (The CURE Investigators 2001)

PICO for the CURE study
Participants Patients within 24 hours of admission with acute coronary syndrome; average age approx. 64; multiple cardiovascular risk factors; patients with risk of bleeding excluded
Intervention Clopidogrel in addition to aspirin and other aspects of standard care
Comparator Aspirin and other aspects of standard care
Outcome Primary endpoint: cardiovascular death, nonfatal MI or stroke; important safety endpoint: major bleed

Was assignment of patients randomized? (1a)

CURE study

After we had obtained written informed consent, patients were randomly assigned to either the clopiogrel group or the placebo group by a central, 24-hour, computerized randomization service. Permuted-block randomization, stratified according to clinical centre, was used.

The CURE Investigators (2001, 495)

Random allocation

  • Random allocation requires a random number generator

  • Random allocation \(\neq\) haphazard allocation, alternate allocation, etc.

  • The unit of random allocation should be the unit of analysis

A health district decides to assess the effectiveness of the implementation of digital hospital services.

  • Implementation Package A is compared to Implementation Package B.

  • The 14 hospitals in the district are randomized to Implementation Package A or B.

  • 28,000 separations from the hospitals in the district are expected throughout the duration of the study.

Block and stratified random allocation

Block random allocation

Allocate participants within a block to ensure similar sizes in each group, e.g. ABBA, BAAB, …

Stratified random allocation

Ensure even numbers of participants with a specific characteristic are randomized to each group. Participants are put in stratified groups and then randomized in blocks.

The CURE study used ‘permuted-block randomization, stratified according to clinical centre’. This ensures (roughly) even distribution between treatment and control in each centre.

Were the groups similar at the start of the trial? (1b)

Table 1, CURE

Assessing similarity at baseline

  • More important with smaller trials

  • The larger the trial, the less likely there will be clinically important differences at baseline

  • Crude statistical comparisons at baseline are not reliable

  • For the difference to be important, the covariate needs to be a possible confounder (and not adjusted for in the analysis)

Aside from the allocated treatment, were groups treated equally? (2a)

CURE Study

A loading dose of clopidogrel (300mg orally) or matching placebo was administered immediately, followed by clopidogrel (75mg per day) or matching placebo for 3 to 12 months (mean duration of treatment, 9 months). Aspirin (recommended dose, 75mg to 325mg daily) was started or continued simultaneously with the study drug. Follow-up assessments occurred at discharge, at one and three months, and then every three months until the end of the study.

The CURE Investigators (2001, 495)

Equal treatment within the study

  • Treatment and follow-up needs to be the same or the differences could bring in biases in some endpoints

  • This can be challenging when the intervention is not a medicine (e.g. cognitive behavioural therapy, acupuncture, ‘mindfulness’, physiotherapy, …)

Haake et al. (2007): Patients with chronic low back pain were randomized to ten 30-minute sessions of acupuncture, sham acupuncture, or conventional therapy (pain medications, physiotherapy). Primary endpoint: treatment response at 6 months.

Response (linked to self-reported low back pain scores):

  • 47.6% acupuncture

  • 44.2% sham acupuncture

  • 27.4% conventional therapy

Were all patients who entered the trial accounted for?—and were they analysed in the groups to which they were randomized? (2b)

Information regarding drop-outs is limited

A total of 21.1 percent of patients in the clopidogrel group discontinued the study medication permanently, as compared with 18.8 percent in the placebo group.

The CURE Investigators (2001, 498)

All analyses were based on the intention-to-treat principle…

The CURE Investigators (2001, 495)

Participant flow

  • How many participants discontinued their allocated treatment?—why?—what did they receive instead?

  • Important to consider flow out of and into the treatment group

  • How were patients that discontinued or lost to follow-up analysed?

CONSORT Diagram

CONSORT Diagram

Intention-to-treat analysis

Intention-to-treat analysis (ITT)

compares treatment outcomes between participants according to treatment allocation (regardless of whether they received the treatment)

Per protocol analysis

A per protocol analysis compares treatment outcomes among participants who adhered to the treatment protocol

  • Random allocation ensures that the treatment groups are comparable at baseline

  • What happens during the trial is another matter:

    • Drop outs: participants allocated treatment who discontinue their allocated treatment

    • Drop ins: participants allocated placebo or control who start taking the experimental treatment

  • The effects of drop out/drop in/loss to follow up is unlikely to be random

How drop out/drop in/loss to follow up is analysed will influence the estimate of treatment effect provided by the study.

Participant flow

The importance of considering patrticipant flow through the study for different analysis types Heritier (2003).

Non-adherence to trial allocation

Starting assumptions

  1. The baseline risk of the event in the population is 10%
  2. The intervention treatment reduces the relative risk of the event by 20%

Consider the following scenario

  1. Non-adherence to treatment allocation occurs equally in both groups
  2. High risk drop-in: in the control group, participants with a higher baseline risk “drop in” to the treatment group

The columns demonstrate the effect of different levels of adherence to treatment allocation on types of analysis. The baseline risk of the event is 10%. The relative risk of the event on treatment is 0.8 (i.e. treatment reduces the relative risk of the event by 20%). Non-adherence effects both groups. In the control group, participants with higher baseline risk, 20%, end up taking the experimental treatment (drop-in).

The following plots look at the same trial under different circumstances. I: nonadherence occurs in both groups, but is independent of baseline risk. II: nonadherence occurs in both groups; participants in the intervention group who drop out have a higher baseline risk (20%). In III an IV, nonahderence is independent of baseline risk. III: nonadherence selectively occurs in the control group (who drop in to treatment). IV: nonadherence selectively occurs in the treatment group.

Heart Protection Study (Heart Protection Study Collaborative Group 2002)

  • The Heart Protection Study randomized \(>\) 20,000 participants with coronary artery disease or diabetes to simvastatin 40mg daily or placebo (starting 1994)

  • Participants allocated simvastatin or placebo could start ‘non-study’ statin if GP thought it indicated

  • Primary endpoint: all-cause mortality (analysed according to intention-to-treat); 12.9% simvastatin v 14.7% placebo

  • Of the participants allocated simvastatin: 82% remained adherent, 3% took a non-study statin, 2% took both

  • Of the participants allocated placebo: by year 5, 32% took a non-study statin

WHI calcium study

  • Jackson et al. (2006) randomized 36,000 women (50–79 years old) to calcium and vitamin D supplementation for approx. 7 years

  • Primary end-point analysed according to intention-to-treat: hip fractures hazard ratio 0.88 (95% CI 0.72–1.08)

  • But also provide a pre-planned sensitivity analysis in the adherent population (censored 6 months after identification of nonadherence)

  • Approx. 60% of participants were adherent (defined as taking at least 80% of study medications)

  • Hip fracture rate among adherers: hazard ratio 0.71 (95% CI 0.52–0.97)

Were measures objective or were the patients and clinicians kept “blind” to which treatment was being received? (3)

CURE Study

We undertook a randomized, double-blind, placebo-controlled trial…

The CURE Investigators (2001, 494)

All primary outcomes and life-threatening and major bleeding complications were adjudicated by persons who were unaware of the patient’s treatment-group assignments.

The CURE Investigators (2001, 495)

Masking

  • If participants, clinicians, people involved in collecting data and/or assessing outcomes know what treatment the participant has received it can influence how they volunteer, describe and/or interpret outcomes

  • Masking is especially important for subjective outcomes (Hróbjartsson and Gøtzsche 2010)

  • Subjective outcomes includes those that rely on participant self-report (e.g. pain, nausea)

Homework!

Have a go at the knowledge-check questions on Blackboard

Reference

Haake, Michael, Han-Helge Muller, Carmen Schade-Brittinger, Heinz D Basler, Helmut Schafer, Christoph Maier, Heinz G Endres, Hans Joachim Trampisch, and Albrecht Molsberger. 2007. “German Acupuncture Trials for Chronic Low Back Pain.” Archives of Internal Medicine 167 (17): 1892–8. https://doi.org/10.1001/archinte.168.9.1011-a.

Heart Protection Study Collaborative Group. 2002. “MRC/BHF Heart Protection Study of Cholesterol Lowering with Simvastatin in 20,536 High-Risk Individuals: A Randomised Placebo-Controlled Trial.” The Lancet 360 (9326): 7–22. https://doi.org/10.1016/S0140-6736(02)09327-3.

Hróbjartsson, Asbjørn, and Peter C Gøtzsche. 2010. “Placebo Interventions for All Clinical Conditions.” Cochrane Database of Systematic Reviews (Online), no. 1: CD003974. https://doi.org/10.1002/14651858.C.

Jackson, Rebecca D, Andrea Z LaCroix, Margery Gass, Robert B Wallace, John Robbins, Cora E Lewis, Tamsen Bassford, et al. 2006. “Calcium Plus Vitamin D Supplementation and the Risk of Fractures.” New England Journal of Medicine 354 (7): 669–83. https://doi.org/10.1056/NEJMoa055218.

The CURE Investigators. 2001. “Effects of Clopidogrel in Addition to Aspirin in Patients with Acute Coronary Syndromes Without ST-Segment Elevation.” New England Journal of Medicine 345 (7): 494–502.